本文旨在通过探索基于神经网络的方法（称为Sun）中的内在不确定性来提高文本到SQL解析的性能。从数据不确定性的角度来看，可以从多个语义等效的问题中学到单个SQL。从以前仅限于一对一映射的方法中不同，我们提出了一个数据不确定性限制来探索潜在的互补语义语义多个语义等效问题（多对一）中的信息，并以减少的虚假关联来学习稳健的特征表示。通过这种方式，我们可以降低学习表示的敏感性并改善解析器的鲁棒性。从模型的不确定性角度来看，神经网络的权重之间通常存在结构信息（依赖性）。为了提高神经文本到SQL解析器的普遍性和稳定性，我们提出了模型不确定性约束，以通过强制执行不同扰动编码网络的输出表示形式来完善查询表示形式，以使其彼此一致。在五个基准数据集上进行的广泛实验表明，我们的方法显着优于强大的竞争对手，并实现了新的最新结果。为了获得可重复性，我们在https://github.com/alibabaresearch/damo-convai/tree/main/main/sunsql上发布代码和数据。

文本到SQL解析是一项必不可少且具有挑战性的任务。文本到SQL解析的目的是根据关系数据库提供的证据将自然语言（NL）问题转换为其相应的结构性查询语言（SQL）。来自数据库社区的早期文本到SQL解析系统取得了显着的进展，重度人类工程和用户与系统的互动的成本。近年来，深层神经网络通过神经生成模型显着提出了这项任务，该模型会自动学习从输入NL问题到输出SQL查询的映射功能。随后，大型的预训练的语言模型将文本到SQL解析任务的最新作品带到了一个新级别。在这项调查中，我们对文本到SQL解析的深度学习方法进行了全面的评论。首先，我们介绍了文本到SQL解析语料库，可以归类为单转和多转。其次，我们提供了预先训练的语言模型和现有文本解析方法的系统概述。第三，我们向读者展示了文本到SQL解析所面临的挑战，并探索了该领域的一些潜在未来方向。

及时调整是将预训练的语言模型调整为下游任务的一种新兴方法。但是，现有的研究主要是为输入序列增加提示。由于中间多头自我注意和馈送网络计算，因此这种方式无法正常工作，从而使模型优化不是很好。因此，我们提出了一种称为“图层调整”的新颖调整方式，旨在在变压器层中添加可学习的参数。具体而言，我们专注于变压器中的馈电网络的图层调整，即FLANing。它将其他单元引入每个馈送网络的隐藏层。我们对公共线索基准进行了广泛的实验。结果表明：1）在几乎所有情况下，我们的FL-tuning tospormports促进了全数据和少量设置下的调整方法。特别是，它在WSC 1.0上的准确性提高了17.93％（全数据设置），而F1上的精度则提高了P-Tuning V2上的Cluener上的精度（几乎没有射击设置）。 2）我们的FL-调整更稳定，收敛速度比P-Tuning V2快约1.17倍。 3）只有大约3％的变压器参数要训练，因此在大多数数据集中进行了微调，并且在几个数据集上的微调（例如，WSC 1.1上的准确性提高了12.9％）。源代码可从https://github.com/genggui001/fl-tuning获得。

长期以来，可以将可以应用于新数据库的文本到SQL解析器的重要性已得到认可，实现此目标的关键步骤是架构链接，即在生成SQL时正确地识别未见列或表的提及。在这项工作中，我们提出了一个新颖的框架，以通过基于PoinCar \'e距离指标的探测程序从大规模预训练的语言模型（PLM）中引起关系结构，并使用诱导的关系来增强基于图的解析器为了更好的模式链接。与常用的基于规则的架构链接方法相比，我们发现探测关系也可以稳健地捕获语义对应关系，即使提及和实体的表面形式不同。此外，我们的探测过程完全不受监督，不需要其他参数。广泛的实验表明，我们的框架在三个基准测试中设定了新的最新性能。我们从经验上验证我们的探测程序确实可以通过定性分析找到所需的关系结构。

最近训练模型通过利用大规模文本语料库来改善神经网络的上下文表示能力，显着提高了各种NLP任务的性能。大型预培训语言模型也已应用于表语义解析的区域。然而，现有的预训练方法没有仔细探索问题与相应的数据库模式之间的明确互动关系，这是揭示其语义和结构对应的关键成分。此外，在架构接地背景下的问知表示学习在预训练目标中受到更少的关注。为了减轻这些问题，本文设计了两种新的预训练目标，将所需的归纳偏差将所需的归纳偏差施加到表前的学习表现-训练。我们进一步提出了一种模式感知课程学习方法来减轻噪声的影响，并以易于努力的方式从预训练数据中学习。我们通过在两个基准，蜘蛛和罢工中进行微调，评估我们预先接受训练的框架。结果表明，与各种基线相比，我们的预训练目标和课程的有效性。

In this paper, we propose a robust 3D detector, named Cross Modal Transformer (CMT), for end-to-end 3D multi-modal detection. Without explicit view transformation, CMT takes the image and point clouds tokens as inputs and directly outputs accurate 3D bounding boxes. The spatial alignment of multi-modal tokens is performed implicitly, by encoding the 3D points into multi-modal features. The core design of CMT is quite simple while its performance is impressive. CMT obtains 73.0% NDS on nuScenes benchmark. Moreover, CMT has a strong robustness even if the LiDAR is missing. Code will be released at https://github.com/junjie18/CMT.

Knowledge graphs (KG) have served as the key component of various natural language processing applications. Commonsense knowledge graphs (CKG) are a special type of KG, where entities and relations are composed of free-form text. However, previous works in KG completion and CKG completion suffer from long-tail relations and newly-added relations which do not have many know triples for training. In light of this, few-shot KG completion (FKGC), which requires the strengths of graph representation learning and few-shot learning, has been proposed to challenge the problem of limited annotated data. In this paper, we comprehensively survey previous attempts on such tasks in the form of a series of methods and applications. Specifically, we first introduce FKGC challenges, commonly used KGs, and CKGs. Then we systematically categorize and summarize existing works in terms of the type of KGs and the methods. Finally, we present applications of FKGC models on prediction tasks in different areas and share our thoughts on future research directions of FKGC.

Few Shot Instance Segmentation (FSIS) requires models to detect and segment novel classes with limited several support examples. In this work, we explore a simple yet unified solution for FSIS as well as its incremental variants, and introduce a new framework named Reference Twice (RefT) to fully explore the relationship between support/query features based on a Transformer-like framework. Our key insights are two folds: Firstly, with the aid of support masks, we can generate dynamic class centers more appropriately to re-weight query features. Secondly, we find that support object queries have already encoded key factors after base training. In this way, the query features can be enhanced twice from two aspects, i.e., feature-level and instance-level. In particular, we firstly design a mask-based dynamic weighting module to enhance support features and then propose to link object queries for better calibration via cross-attention. After the above steps, the novel classes can be improved significantly over our strong baseline. Additionally, our new framework can be easily extended to incremental FSIS with minor modification. When benchmarking results on the COCO dataset for FSIS, gFSIS, and iFSIS settings, our method achieves a competitive performance compared to existing approaches across different shots, e.g., we boost nAP by noticeable +8.2/+9.4 over the current state-of-the-art FSIS method for 10/30-shot. We further demonstrate the superiority of our approach on Few Shot Object Detection. Code and model will be available.

Graph Neural Networks (GNNs) have shown satisfying performance on various graph learning tasks. To achieve better fitting capability, most GNNs are with a large number of parameters, which makes these GNNs computationally expensive. Therefore, it is difficult to deploy them onto edge devices with scarce computational resources, e.g., mobile phones and wearable smart devices. Knowledge Distillation (KD) is a common solution to compress GNNs, where a light-weighted model (i.e., the student model) is encouraged to mimic the behavior of a computationally expensive GNN (i.e., the teacher GNN model). Nevertheless, most existing GNN-based KD methods lack fairness consideration. As a consequence, the student model usually inherits and even exaggerates the bias from the teacher GNN. To handle such a problem, we take initial steps towards fair knowledge distillation for GNNs. Specifically, we first formulate a novel problem of fair knowledge distillation for GNN-based teacher-student frameworks. Then we propose a principled framework named RELIANT to mitigate the bias exhibited by the student model. Notably, the design of RELIANT is decoupled from any specific teacher and student model structures, and thus can be easily adapted to various GNN-based KD frameworks. We perform extensive experiments on multiple real-world datasets, which corroborates that RELIANT achieves less biased GNN knowledge distillation while maintaining high prediction utility.

This paper focuses on designing efficient models with low parameters and FLOPs for dense predictions. Even though CNN-based lightweight methods have achieved stunning results after years of research, trading-off model accuracy and constrained resources still need further improvements. This work rethinks the essential unity of efficient Inverted Residual Block in MobileNetv2 and effective Transformer in ViT, inductively abstracting a general concept of Meta-Mobile Block, and we argue that the specific instantiation is very important to model performance though sharing the same framework. Motivated by this phenomenon, we deduce a simple yet efficient modern \textbf{I}nverted \textbf{R}esidual \textbf{M}obile \textbf{B}lock (iRMB) for mobile applications, which absorbs CNN-like efficiency to model short-distance dependency and Transformer-like dynamic modeling capability to learn long-distance interactions. Furthermore, we design a ResNet-like 4-phase \textbf{E}fficient \textbf{MO}del (EMO) based only on a series of iRMBs for dense applications. Massive experiments on ImageNet-1K, COCO2017, and ADE20K benchmarks demonstrate the superiority of our EMO over state-of-the-art methods, \eg, our EMO-1M/2M/5M achieve 71.5, 75.1, and 78.4 Top-1 that surpass \textbf{SoTA} CNN-/Transformer-based models, while trading-off the model accuracy and efficiency well.